Ensembled sparse‐input hierarchical networks for high‐dimensional datasets

In high-dimensional datasets where the number of covariates far exceeds observations, most popular prediction methods make strong modeling assumptions. Unfortunately, these struggle to scale up in model complexity as observations grows. To this end, we consider using neural networks because they span a wide range capacities, from sparse linear models deep networks. Because are notoriously tedious tune and train, our aim is develop convenient procedure that employs minimal hyperparameters. Our method, Ensemble by Averaging Sparse-Input hiERarchical (EASIER-net), only two L1-penalty parameters, one controls input sparsity another for hidden layers nodes. EASIER-net selects true support with high probability when there sufficient evidence; otherwise, it performs variable selection uncertainty quantification, strongly correlated selected at similar rates. On large collection gene expression datasets, achieved higher classification accuracy fewer genes than existing methods. We found adaptively complexity: fit was information learn nonlinearities interactions logistic smaller less information.